Estimating the Accuracy of Anal Cytology in the Presence of an Imperfect Reference Standard

BACKGROUND The study aim is to estimate sensitivity and specificity of anal cytology for histologic HSIL in analyses adjusted for the imperfect biopsy reference standard. METHODS AND PRINCIPAL FINDINGS Retrospective cohort study of an anal dysplasia screening program for HIV infected adults. We estimated the prevalence of histologic HSIL by concurrent cytology category and the associated cytology ROC area. Cytology operating characteristics for HSIL were estimated and adjusted for the imperfect reference standard by 3 methodologies. The study cohort included 261 patients with 3 available measures: (1) referral cytology; (2) HRA cytology; and (3) HRA directed biopsy. The prevalence of biopsy HSIL varied according to the concurrent HRA cytology result: 64.5% for HSIL or ASC-H, 12.6% for LSIL, 10.9% for ASCUS, and 6.3% for no abnormality. The cytology ROC area was 0.78. The observed prevalence of HSIL was 37% (referral cytology), 24% (HRA cytology), and 24% (HRA biopsy). Unadjusted estimates of sensitivity and specificity of cytology were 0.66 and 0.90, respectively. Adjusted estimates varied from 0.47-0.89 (sensitivity) and 0.89-1.0 (specificity). CONCLUSIONS Analysis of a single dataset yields widely different estimates of anal cytology operating characteristics that depend on difficult to verify assumptions regarding the accuracy of the imperfect reference standard.